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The Achieving the Dream (ATD) initiative works with more than 100 community 
colleges across the United States with the specific goal of increasing student success. 
Together, Achieving the Dream colleges graduate or transfer close to 250,000 
students a year. With just a 5 percent increase in graduation rates, we can positively 
impact the lives of an additional 12,500 students each year — or 62,500 students over 
five years. That’s 62,500 additional students achieving a life-changing goal because 
of the work to which we are committing ourselves today. 


We learned during the first five years of Achieving the 
Dream that attaining a large-scale increase in student success 
is dependent on several things, the most important being the 
collection, analysis, and use of data. We need data to inform 
us which students are most at risk of not succeeding. We 
need data to indicate why specific student groups are not 
succeeding. We need data to show us which components 
of our interventions have worked or have not worked. 

And, we need data to determine the extent to which our 
interventions have led to increases in student success. 

Using data in such ways is not traditional practice for 
many colleges. In fact, some might argue that it would 
require fundamental change for a college to collect, 
analyze, summarize, and use data in this manner. Indeed, 
fundamental change is exactly what Achieving the Dream 
is all about. 

Facilitating institutional change is not easy. It requires 
new organizational language, practices, interactions, and 
relationships — things that do not happen overnight. This 


guide is designed to achieve two objectives: (a) offer new 
language to clarify the use and purpose of various types 
of data, and (b) demonstrate how colleges can engage in 
the practice of identifying (or diagnosing) the underlying 
factors impeding student success and address such factors 
in the design of specific interventions or policy changes. 

Gaining Clarity About Data: The Four 
Components of a Culture of Evidence 

Early in the Achieving the Dream initiative, it was not 
uncommon for faculty, staff, and administrators to become 
overwhelmed and confused about the data that existed 
on their campuses. There was either too much data or 
not enough. There were data about all students, but not 
specific groups. There were data that indicated problem 
areas, but no data to explain why. Occasionally, a report 
containing data was presented at college meetings, but 
few knew what it meant or how they were supposed 
to use it. In short, many colleges were struggling with 
creating a coherent culture of evidence. In an effort to 
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Table 1 

The Four Components of a Culture of Evidence 


Component One 


Component Two 


“Why” 


Component Three 


“Intervention’ 


Component Four 


“What’s Wrong” 

Use disaggregated longitudinal cohort 

data to determine: 

1 ) Which student groups are less 
successful than others (i.e., identify 
equity gaps in student success)? 

2) Which high enrollment courses 
have the lowest success rates? 


Collect, analyze, and use data from 
other sources (focus groups, surveys, 
literature reviews) to identify the under- 
lying factors (barriers or challenges) 
impeding student success. 


Use data from Component Two to 
design new interventions, or revise 
current ones, to effectively address 
the underlying factors impeding 
student success. 

Review and consider changes to 
existing college practices and policies 
that impact the underlying factors 
impeding student success. 


“Evaluation and Modification” 
Collect, analyze, and use evaluation 
data to answer: 

1 ) To what extent did the interventions 
(including policy changes) effectively 
address the underlying factors imped- 
ing student success? 

2) To what extent did the interventions 
increase student success? 

Make modifications based on 
evaluation results. 


facilitate clarity, the following framework, titled “The 
Four Components,’’ describes how colleges can use data 
to increase success. 

Component Number One: 

Using Disaggregated Longitudinal Cohort Data 
to Find Out “What’s Wrong?” 

In the colleges’ efforts to increase student success, it 
became clear that it was necessary for everyone to 
understand the types of data needed and exactly how such 
data was to be used. The first step involves collecting and 
analyzing disaggregated longitudinal cohort data. A 
companion guide provides a summary of the “ins and outs” 
of collecting, analyzing, summarizing, and using this initial 
type of data. Simply put, the purpose of disaggregated 
longitudinal cohort data is to answer the question: “What’s 
wrong?” With this type of data, a college can identify: (a) 
which student groups are less successful than others, in 
terms of the five Achieving the Dream Student 
Performance Measures (persistence; successful completion 
of developmental education; successful completion of a 
course with a “C” or better; successful completion of 
gatekeeper courses; and successful completion of a degree, 
certificate, or transfer), and (b) which high-enrollment 
courses have the lowest completion rates. Disaggregated 
longitudinal cohort data also allow a college to determine 
where it should focus its time and resources. For example, 
should a college focus on: (a) a specific course, (b) a specific 
population as it relates to a specific course, (c) persistence 
for all students, or (d) persistence as it relates to a specific 
student population? 


Component Number Two: 

Using Data to Answer the “Why” Question 
(Diagnosis) 

Early on, we discovered that it was not sufficient to simply 
identify “What’s wrong?” (for example, which student 
groups are most at-risk of not succeeding); we also needed 
to understand and identify why. That required collection 
of a second set of data. Why are particular student groups 
less successful than others? Specifically, what underlying 
factors (barriers or challenges) impede their success? 
Without identifying specific underlying factors, colleges 
found themselves with limited capacity to design 
interventions that would be effective in increasing student 
success. Even when moderate gains were achieved, 
colleges were not able to identify or isolate the specific 
components of an intervention or set of interventions that 
contributed to increases in student success. 

Component Number Three: 

Addressing the Underlying Factors Impeding 
Student Success through New or Revised 
Interventions or Policy Changes 

After collecting a second set of data to answer the 
“why” question, a college is then ready to revise 
existing interventions or create new ones to address the 
specific underlying factors impeding student success. 
Additionally, a college needs to review and consider the 
impact of existing college policies on the underlying 
factors impeding student success. 
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Component Number Four: 

Assessing Impact — Evaluating Interventions 
to Increase Student Success 

Next, the college must evaluate its interventions (see 
the companion guide “Evaluating Student Success 
Interventions”). This data enables colleges to determine: 

(a) the effectiveness of an intervention or policy change in 
addressing the underlying factors impeding student success, 
and (b) the extent to which an intervention or policy change 
led to increases in student success. 

Summarizing the Four Components 
of a Culture of Evidence 

As summarized in Table 1, there are four components 
to using data to create and sustain a culture of evidence. 
Component Number One involves the use of longitudinal, 
disaggregated cohort data to answer the “what’s wrong” 
question as it relates to specific student success outcomes. 
Component Number Two entails using a second set of data 
to answer the “why” question and identify the underlying 
factors that impede student success. Component Number 
Three involves designing an intervention — or making 
a policy change — that addresses the underlying factors 
impeding student success. Finally, Component Number 
Four includes evaluating: (a) the effectiveness of the 
college’s intervention or policy change in addressing the 
underlying factors and (b) the extent to which the college’s 
intervention or policy change increased student success. 

Following are examples of how two Achieving the Dream 
colleges used data to identify (or diagnose) underlying factors 
impeding student success and effectively addressed them in 
the design of a specific intervention. 

A Focus on Diagnosis 

Prior to Achieving the Dream, a handful of participating 
colleges had collected, analyzed, and used disaggregated 
longitudinal cohort data. These colleges knew, for example, 
that the course completion rates of male students taking 
developmental English courses at their institution were 
quite low. What they didn’t know was why. Moreover, they 
quickly realized that their capacity to significantly increase 
course completion rates for males in developmental English 
would be continually hampered until they had the data 
that indicated the specific underlying factors impeding the 
success of these students. Acquiring data to identify such 
underlying factors was no simple task. Questions emerged, 
such as: (a) What data do we need to identify the underlying 


factors? (b) Can we use existing data or do we need to collect 
new data? (c) Do we administer a survey or do we conduct 
focus groups? (d) If we conduct a survey, can we use an 
existing instrument or do we need to develop our own? (e) 
Should we conduct a review of the literature, and if so, who 
will be responsible for completing and sharing it? 

What is clear to everyone in the Achieving the Dream 
initiative is that we must answer the “why” question and 
identify the specific causes or underlying factors if we 
are to significantly increase student success. The paths 
colleges can take to identify underlying factors are many. 

In the next section, I present three different approaches or 
methods to answering the “why” question: (a) conducting 
focus groups, (b) completing a review of the literature, and 
(c) using existing survey data. Ideally, colleges would use 
multiple sources of data to triangulate the specific underlying 
factors. Other approaches to identifying underlying factors 
include using transcript analysis (Hagedorn et. al, 2003) and 
assessment of student learning outcomes. 

Conducting Focus Groups to Identify Underlying Factors 

Conducting focus groups — in the context of Achieving the 
Dream — involves facilitating a series of focused, small- 
group conversations with the goal of identifying common 
responses related to specific underlying factors impeding 
student success. The benefits of conducting focus groups 
include: (a) producing a local, campus-based understanding 
of underlying factors impeding student success; (b) allowing 
students, faculty, and staff members an opportunity to have 
their voices heard; and (c) increasing the capacity of the 
college to use multiple sources of data when identifying 
underlying factors impeding student success. If faculty and 
staff members are trained in conducting focus groups, the 
college also expands its capacity to conduct and use focus 
group data in the future. 

There are particular challenges to conducting focus groups, 
such as determining: (a) which groups to target in a focus 
group; (b) what questions to ask in a focus group; (c) who 
will conduct the focus group and analyze, summarize, and 
report the data; (d) how to gain trust and confidence in focus 
group data; (e) and how to use focus group data to design 
interventions to increase student success. Each of these 
challenges is addressed in the case below. Please note: This 
guide is intended to provide a brief overview of specific approaches 
to identifying underlying factors impeding student success. 

The following material is not presented as a training guide for 
conducting effective focus groups. 
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Using Focus Group Data to Design Student Success 
Interventions at Tulsa Community College 

Using focus groups to identify underlying factors After 
compiling, reviewing, and discussing Tulsa Community 
College’s (TCC) disaggregated longitudinal cohort 
data (Component Number One), the college achieved 
consensus concerning specific problem areas. Based on 
the data, TCC’s Achieving the Dream Core Team chose 
to prioritize its time and resources to address four areas: 

(a) fall-to-spring persistence for all first-time-in-college 
students, (b) fall-to-spring and fall-to-fall persistence for 
first-time-in-college African-American males, (c) course 
completion rates for students enrolled in developmental 
reading courses, and (d) course completion rates for 
students enrolled in developmental math courses. The 
college developed a plan to address each of these areas 
over a four-year period. Increasing the persistence rate for 
all first-time-in-college students was addressed first. 

Having identified its priority areas, the college’s Data 
Team, in consultation with the college leadership team, 
chose to conduct focus groups as the primary method of 
identifying the underlying factors impeding successful fall 
semester persistence for all first-time-in-college students. 
To expand TCC’s research capacity, the Data Team 
recruited and identified five faculty members and one 
administrator to participate in a two-day training session 
(Gonzalez, 2007) on how to collect, analyze, summarize, 
and use focus group data. This training was largely based 
on the focus group approach developed by Raymond 
V. Padilla (1999; 2008; Padilla, Trevino, Gonzalez, & 
Trevino, 1997). Padilla’s approach is highly structured 
and allows for quick analysis, interpretation, and use of 
the data. The approach also can be used to identify 
underlying factors impeding student success in many 
areas, such as persistence or the successful completion 
of a particular course. 


The questions guiding TCC’s focus on persistence 
included: (a) What barriers or challenges do first-time- 
in-college students experience in their first semester? 

(b) What do students know or do that enables them 
to overcome specific barriers or challenges in the first 
semester? (c) What changes can the college make to 
eliminate, reduce, or assist students in overcoming specific 
barriers or challenges in the first semester? (d) What does 
the college do currently that effectively helps students 
overcome specific barriers or challenges in the first 
semester? 

Prior to conducting focus groups, the Data Team met 
to discuss and decide how many focus groups should 
be conducted to produce a quality data set for analysis. 
Given that Tulsa Community College serves students on 
four geographically separated campuses, the Data Team 
chose to conduct twelve student focus groups — three 
on each campus. The Data Team members wanted to 
ensure that the focus group data would allow them to 
identify campus-specific differences. The Data Team 
also chose to conduct eight additional focus groups — two 
on each campus — that targeted TCC faculty and staff 
members. Finally, the team developed a comprehensive 
recruitment plan that produced: (a) an adequate number 
of participants for each focus group and (b) a group of 
students that mirrored the demographic characteristics of 
the TCC student population. 

After conducting 20 focus groups on all four campuses, the 
Data Team spent the next two months analyzing the data 
and identifying the common underlying factors impeding 
fall semester persistence (see Table 2 for timeline). Using 
the constant comparative method (Glaser & Strauss, 1967) 
for data analysis, the Data Team identified six common 
underlying factors impeding successful semester-to- 
semester persistence. 


Table 2 

TCC Timeline for Completing the Four Components of Increasing Student Success 

2007 2008 


Oct. 

Nov. - Dec. 

Jan. - Feb. 

March - June 

July 

Aug. - Dec. 

Component One 

Component Two 


Components Three and Four 


Identification of Problem 
Areas 

Focus Group Training and 
Conduction of 20 Focus 
Groups 

Data Analysis and 
Identification of Common 
Underlying Factors 

Discussion and Use of Focus 
Group Data to Design College-wide 
Intervention: Student Success Course 

Development of Common Course 
Learning Outcomes and Assessment 
and Evaluation Plan 


Implementation of Intervention 
Assessment and Evaluation 
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Table 3 


Underlying Factors Impeding Successful Fall Semester Persistence at TCC 


Adjusting to 
college 

Balancing school 
and life 

Difficulty choosing 
courses 

Communication 
issues with 
instructors 

Textbook issues 

Tulsa Achieves 

Implementation 

Issues 

Too much freedom 

Getting used to being on 
your own 

Having lots of stress 

Adjusting to different kinds 
of students 

Time and effort to do well 
in courses (very different 
from high school) 

Balancing needs of family, 
study time, and personal 
time 

Juggling home and school 
responsibilities 

Scheduling around your 
children 

Not knowing which 
courses are needed for 
your major 

Not knowing what 
courses to select and 
which courses transfer; 
need more guidance in 
planning class schedule 

Limited choices of some 
required classes 

Not getting feedback 
on assignments and not 
knowing how you are 
doing in class 

Unable to find out how 
you are progressing in 
class 

Instructors don't know 
their students 

Students and teachers are 
using different editions of 
the textbook 

TCC Campus Store didn't 
have the required books 

Purchase on-line course 
materials only at NE 
campus 

Inadequate bookstore 
refund policy 

Paying for textbooks 

Financial aid processing 

Information about TA 
constantly changing 
and sometimes was 
inaccurate 


The common underlying factors reported in the focus 
groups included: (1) adjusting to college, (2) balancing 
school and life, (3) difficulty choosing courses, (4) 
communication issues with instructors, (5) textbook 
issues, and (6) Tulsa Achieves implementation issues. 
Tulsa Achieves is a recruitment and retention program 
for local Tulsa residents. The Data Team used specific 
language from the student focus groups to define the 
six themes noted above. The six underlying factors, and 
supporting language, are found in Table 3. Finally, the 
Data Team reported that there were no underlying factors 
that were unique to a specific campus. 

Using focus group findings to design a college-wide 
intervention After collecting, analyzing, and summarizing 
the focus group data, the Data Team used the findings 
to facilitate a discussion with the college leadership team 
about how to design an intervention that would address 
most or all of the identified underlying factors impeding 
fall semester persistence. After much deliberation, the 
college leadership team chose to convene a group of 
faculty from all four campuses to completely redesign 
an existing student success course to address four of the 
six underlying factors. This team of faculty members 
met over the summer and accomplished the following: 

(a) used the focus group data to produce common course 
learning outcomes (objectives) for the student success 
course, (b) gained consensus on using a common textbook 


for the course, (c) identified additional faculty to teach 
the course, which expanded the available fall sections 
from six to sixty-four, (d) conducted summer training 
for all faculty teaching the course, and (e) designed an 
evaluation and learning outcomes assessment plan, which 
was implemented in the fall semester. Table 4 illustrates 
how the common course learning outcomes addressed 
four of the six underlying factors impeding successful fall 
semester persistence. 

Results: The impact of TCC’s redesigned student 
success course on fall-to-spring persistence rates 

Sixty-four sections of TCC’s redesigned student success 
course were offered in the Fall 2008 semester. Prior to 
TCC’s involvement in Achieving the Dream, the college 
offered approximately six sections of a student success 
course each semester. Of the nearly 3,000 first-time-in- 
college students enrolled in the fall semester of 2008, 

561 enrolled in the redesigned student success course. 
The fall-to-spring persistence rate for first-time-in-college 
students enrolled in TCC’s redesigned student success 
course was 81 percent, compared with 63 percent for 
other first-time-in-college students. TCC’s faculty is 
now reviewing the learning outcomes assessment data to 
determine the relationship between a specific learning 
outcome and subsequent persistence. 
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Table 4 


Linking Focus Group Findings with TCC Revised Student Success Course 


Underlying Factors Impeding TCC Student Success Course (Common Course Learning Outcomes) 
Persistence 


Underlying Factors Impeding 
Persistence 

TCC Student Success Course (Common Course Learning Outcomes) 

Adjusting to college 
■ Too much freedom 

1 ) Use personal and social strategies to succeed, enjoy the college experience, and become involved in college 
and community activities. 

■ Getting used to being on your own 

2) Identify and apply college and academic terminology. 

■ Having lots of stress 

3) Construct short-term und lony-lenn yools, bolonciny personol skills, inleresls, peisonolity, und vqIugs. 

■ Adjusting to different kinds of students 

4) Construct and monitor weekly/monthly time plans to balance work, school, family, and social activities. 

■ Time and effort to do well in courses 


(very different from high school) 

6) Analyze and modify study techniques and behavior patterns to successfully complete homework, reading assignments, 
exams, and special projects. 


7) Apply critical and creative thinking skills to identity and solve academic and social problems. 


9) Demonstrate transference of skills learned from other coursework during the semester. 

Balancing school and life 

3) Construct short-term and long-term goals, balancing personal skills, interests, personality, and values. 

■ Balancing needs of family, study time. 

4) Construct and monitor weekly/monthly time plans to balance work, school, family, and social activities. 

and personal time 

5) Locate and apply college resources and support systems and incorporate these into the learning process. 

■ Juggling home and school responsibilities 

■ Scheduling around your children 

6) Analyze and modify study techniques and behavior patterns to successfully complete homework, reading assignments, 
exams, and special projects. 


7) Apply critical and creative thinking skills to identity and solve academic and social problems. 


9) Demonstrate transference of skills learned from other coursework during the semester. 

Difficulty choosing courses 

2) Identify and apply college and academic terminology. 

■ Not knowing which courses are needed for 

3) Construct short-term and long-term goals, balancing personal skills, interests, personality, and values. 

your major 

4) Construct and monitor weekly/monthly time plans to balance work, school, family, and social activities. 

■ Not knowing what courses to select and 
which courses transfer; need more 
guidance in planning class schedule 

■ Limited choices for some required classes 

7) Apply critical and creative thinking skills to identify and solve academic and social problems. 

Communication issues with instructors 
■ Not getting feedback on assignments and 

1 ) Use personal and social strategies to succeed, enjoy the college experience, and become involved in college 
and community activities. 

not knowing how you are doing in class 

3) Construct short-term and long-term goals, balancing personal skills, interests, personality, and values. 

■ Unable to find out how you are progressing 
in class 

5) Locate and apply college resources and support systems and incorporate these into the learning process. 

■ Instructors don't know their students 


Textbook issues 

■ Students and teachers are using 
different editions of the textbook 

■ TCC Campus Store didn't have the required 
books 

■ Purchase on-line course materials only at 
NE campus 

■ Inadequate bookstore refund policy 

TCC currently discussing strategy to address textbook issues. 

Tulsa Achieves Implementation Issues 

■ Paying for textbooks 

■ Financial aid processing 

■ Information about TA constantly changing 
and sometimes was inaccurate 

TCC addressed Tulsa Achieves Implementation Issues with a different intervention. 
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Completing a Review of the Literature to 
Identify Underlying Factors 

For several decades, researchers have written articles on 
the barriers impeding the success of community college 
students. Unfortunately, the vast majority of these studies 
never find their way onto a community college campus. 
Few can make the argument that the model of scholars 
producing knowledge and practitioners consuming 
knowledge is working. 

There are at least two primary challenges to using research 
articles in our work to increase student success. First, most 
research articles are not written for practitioners. Their 
primary audience is other researchers. As a consequence, 
community college faculty and staff members must make 
their way through abstract concepts and complicated 
methodologies to locate the nugget of information that 
may help them serve students better. Second — and even 
more challenging than the first — faculty and staff already 
are overextended and simply do not have the time to read 
and summarize the primary findings of several research 
articles. Other challenges include having the ability to 
assess the quality of a research article, and transforming 
the primary findings of a literature review into actionable 
information. Those challenges are addressed in the 
following case. 

Using a Review of the Literature at Tulsa 
Community College 

As noted earlier, one of Tulsa Community College’s 
priorities was to increase the persistence rates of African- 
American males. During a college -wide presentation of 
the disaggregated longitudinal cohort data, a group of 
TCC African-American male faculty and staff members 
reacted strongly to the data. It was clear that African- 
American male students persisted at a much lower rate 
than all other students. Their reaction to the data turned 
into concerned discussion, followed by planned action. 
This initial group of African-American male faculty and 
staff recruited other African-American male professionals 
from throughout the college and met to discuss what 
they could do to increase the success rates of African- 
American male students at Tulsa Community College. 
They adopted the Four Components approach to ensure 
that their plan would be data-driven. They had the data 
that answered the “what’s wrong” question. Now, they 
needed additional data to identify why. 


The group began its work by discussing and identifying 
the challenges and barriers they overcame to complete 
their undergraduate degrees. This information was a good 
start, but the group knew it needed additional data to 
attain a comprehensive understanding of the underlying 
factors impeding the success of African-American males at 
Tulsa Community College. As a consequence, the group 
chose to conduct a review of the literature on African- 
American males in community colleges, and focus groups 
with TCC African-American male students. 

Several issues were addressed in the group’s plan to 
conduct a review of the literature. First, the members 
discussed the criteria to be used to determine the quality 
of a research article and the trustworthiness of its findings. 
The criteria included using: (a) peer-reviewed research 
articles published in reputable journals or by reputable 
organizations, (b) research reports published by reputable 
organizations, and (c) published works consistently cited 
in the literature. They also identified group members who 
felt comfortable with various research methodologies and 
would be willing to conduct a review of the literature. 
Next, they developed a plan to secure release time for the 
individuals conducting the literature review. They also 
were in agreement that the findings from the literature 
review would be used to support or confirm the findings 
from the focus groups, which would be conducted in 
the Fall 2009 semester. Finally, they discussed and 
developed a template that would be used to transform the 
primary findings of the literature review into actionable 
information. 

TCC’s review of the literature on African-American males 
in community colleges is nearly complete at this writing. 
Preliminary findings were presented to the group using a 
newly developed template. Table 5 shows the template 
used to transform the literature review findings into 
actionable information. The template includes designated 
space to capture and compare the literature review 
findings with the findings from the focus groups and the 
specific intervention components. A full citation for each 
research article can be found following the template. 
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Table 5 


Linking Literature Review Findings with Interventions to Increase Student Success: 

A Focus on African-American Males in Community Colleges — A Working Document 



Challenges/Barriers for AA Males 


Literature Review Citations 

Literature Review Findings 

Focus Group Findings 

Intervention Components 

Cuyjet (1997) 
Flowers (2006) 

Less engaged with the academic 
experience than other students 

To be completed 

To be designed 

Cuyjet (1997) 

Less likely to take detailed notes than 
other students 

To be completed 

To be designed 

Cuyjet (1997) 

Less time devoted to writing and revising 
essays than other students 

To be completed 

To be designed 

Rendon (2000) 
Flowers (2006) 

There is a sense of disconnectedness 
from the social aspect of college life. 

To be completed 

To be designed 

St. John, Cabrera, Nora, & Asker 
(2000) 

Costs of education, costs of books and 
materials 

To be completed 

To be designed 

Jordan (2008) 




Jordan (2008) 

Stereotypes exist that make AA males 
question their ability to be scholars. 

To be completed 

To be designed 

Adelman (2006) 

Hagedorn, Maxwell, & Hampton 
(2001) 

AA males who take more than 4 units 
during the summer term prior to first 
semester have a 78.2% degree 
completion rate. AA males who did not 
enroll in summer term have a 21 .2% 
degree completion rate. 

To be completed 

To be designed 

Dawson-Threat (1 997) 
Quaye & Harper (2007) 
Ellis (2004) 

Jordan (2008) 

Successful class experience for AA males 

is related to three characteristics: 

1 ) 'Safe space' exists to express 
personal experiences. 

2) Classroom culture promotes the 
understanding of difference. 

3) Course provides opportunities 
for Black males to explore 'Black 
Manhood." 

To be completed 

To be designed 

Brown (2006) 
Ellis (2004) 
Jordan (2008) 

Peer support, particularly with other 
Black males, had a positive impact on 
their college experience and sense of 
engagement. 

To be completed 

To be designed 

Harper (2006) 




Bonner & Bailey (2006) 
Jordan (2008) 

Family units and sense of responsibility 
to family serve as key sources of support 
for AA males. 

To be completed 

To be designed 

Hagedorn, Maxwell, & Hampton 
(2001) 

Jordan (2008) 

Bailey, Jenkins, & Leinbach (2005) 

Goal setting and commitment to 
goal has positive impact on college 
experience. 

To be completed 

To be designed 
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Using Survey Data to Identify 
Underlying Factors 

Survey data are abundant on many community college 
campuses. Colleges routinely administer surveys, such as 
the Community College Survey for Student Engagement 
(CCSSE), the Student Satisfaction Inventory (SSI), 
and the HERI Faculty Survey, as well as campus-based 
satisfaction surveys and student surveys of instructor 
effectiveness. The benefits to using survey data include: 
(a) identifying the perceptions or attitudes of a large 
number of students, faculty, or staff members, (b) having 
a level of confidence in the survey results, due to the 
large sample size, (c) identifying trends that may have 
a positive or negative impact on student success, (d) 
helping to test the ability to generalize hypotheses with 
a representative sample of respondents, and (e) receiving 
results in a relatively short time frame, especially with the 
use of on-line surveys. The primary limitations of survey 
data include: (a) not being able to identify the contextual 
factors that may impact the survey results, and (b) not 
being able to probe responses to a particular question. 
These two shortcomings can be addressed, in part, 
through the use of focus groups, as discussed earlier. The 
challenge for most colleges is finding a way to use survey 
data to identify specific underlying factors impeding 
student success. The following case highlights how South 
Texas College used the results from their survey data to 
confirm the identification of underlying factors impeding 
student success in the classroom. 
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Table 6 


Top 1 0 Barriers to Successful Persistence at 
South Texas College, 2005 and 2007 


2005 Barriers Study 

2007 Barriers Study 

Money 

Instructor Issues 

Lack of Information 

Child/Family Issues, Responsibilities 

Work/Job/Time Management 

Lack of Course Offerings/Times Offered 

Lack of Adequate Facilities/Equipment 

Placement in Developmental 

Child/Daycare/Family Issues 

Lack of Course Offerings/Times Offered 

Placement in Developmental 
Courses/TFIEA 

Courses/TFIEA 

Work/Job/Time Management 
Issues with STC Personnel 
Money 

Lack of Access to Technology/Internet 

Lack of Information 

Instructor Issues 

Lack of Adequate Facilities/Equipment 

Issues with STC Personnel 

Lack of Access to Technology/Internet 


Using Survey Data at South Texas College 

Comparing the results of focus groups in two time 
periods South Texas College (STC), located in Texas’ 

Rio Grande Valley, serves more than 21,000 students on 
five different campuses. After reviewing and discussing 
their disaggregated longitudinal cohort data, STC chose to 
address the fall-to-spring and fall-to-fall persistence rates 
for all first-time-in-college, full-time students. Twenty- 
five student focus groups were conducted in the fall of 
2005 that identified, in rank order, the top 10 barriers 
to persistence, and STC used the data to redesign its 
approach to academic advising for all first-time-in-college 
students. The findings from the 2005 focus group study 
are summarized in Table 6. 

Two years later, STC decided to conduct a follow-up 
study on the barriers to student persistence, in large part 
to determine if the college’s interventions eliminated 
or reduced the frequency of the barriers identified in 
the 2005 study. The findings from the 2007 focus group 
study also are summarized in Table 6. What the college 
discovered after comparing the list of top 10 barriers from 
the 2005 study with those in the 2007 study was that 


the top four barriers in the 2005 study were much less 
prevalent in the 2007 study. This provided one source 
of evidence that the interventions designed to decrease 
the barriers related to money, lack of information, and 
time management were working. In contrast, the lower- 
ranked barriers in the 2005 study were found to be higher 
on the list in the 2007 study. Of particular interest and 
concern was the barrier related to “instructor issues.” In 
the 2005 study, “instructor issues” was ranked 9th out 
of the 10 barriers. In the 2007 study, it was ranked first. 
Faculty members were both concerned and surprised by 
the 2007 list of barriers. Many questioned the validity 
of the findings from the 2007 study; others wanted to 
know, specifically, what the students meant in terms of 
“instructor issues.” 

South Texas College responded to the results of the 2007 
study, particularly as it concerned “instructor issues,” 
by convening a group of faculty members to discuss 
and review all available data related to faculty/student 
interactions. This faculty team, along with the Director 
of Institutional Research, accomplished the following 
tasks over a three-month period: They (a) disaggregated 
“instructor issues” from the 2007 barriers study, (b) 
reviewed survey results from CCSSE, CGFSSE, HERI 
Faculty Survey, and Student Evaluations of Faculty 
and determined whether the survey data triangulated 
with the “instructor issues” finding from the 2007 
focus group study, (c) recommended interventions to 
address “instructor issues” as validated through the 
data triangulation process, and (d) developed a 2009 
“Instructor Issues” follow-up study. Discussion about 
each action step follows. 

Disaggregating “instructor issues” After reviewing 
the raw data from the 2007 focus group study, the faculty 
team was able to identify sub-themes and quotes to 
further define the finding “instructor issues.” Sub-themes 
for “instructor issues” included: (a) instructor attitudes, 

(b) quality of instructors, (c) instructor expectations, 

(d) instructor spoken language and accents, and 

(e) instructor practices and course inconsistencies. 

Table 7 includes a list of sub-themes and quotes for the 
finding “instructor issues.” 
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Table 7 


Sub-themes and quotes for “Instructor Issues” finding from 2007 focus group study 


Instructor Attitudes 

Quality of Instructors 

Instructor 

Expectations 

Instructor Spoken 
Language and 
Accents 

Instructor Practices 
and Course 
Inconsistencies 

Some don't care, only on 
payroll; unhelpful, subjective 
faculty; treatment childish 
and unfair 

Some professors are rude/they 
embarrass you/they are rude 
and mean, attitude discouraging, 
intimidating and impatient 

Lack of qualified instructors 
Instructors do not teach/lecture 
Classes are not interesting 

Teach at too high of a level/ 
classes are too advanced 

Teachers expect too much 

Not enough in-class learning, 
expected to learn on our own 

Communication barriers 
Lack of understanding 

Experience base and text 
lectures are not equivalent 

Irrelevant homework 

Communication problem 
outside of class 


Triangulation with survey data. Next, faculty members 
reviewed the survey results from CCSSE, CCFSSE, 
HERI Faculty Survey, and the Student Evaluations 
of Faculty to determine if the survey data triangulated 
with the focus group finding “instructor issues.” They 
discovered that both the CCSSE and the Student 
Evaluations of Faculty survey results supported the 
finding “instructor issues” as it related to instructor 
attitudes, quality, expectations, practices, and course 
inconsistencies. They also noted significant differences 
between student and faculty perceptions with regard 
to: (a) receiving prompt feedback from instructors 
on performance (89 percent faculty vs. 49 percent 
students); (b) discussing grades and assignments with 
instructor (70 percent faculty vs. 48 percent students); 
and (c) faculty encouraging students to ask questions 
in class (96 percent faculty vs. 72 percent students). 

Recommended interventions to address “instructor 
issues” Having found survey data to support the focus 
group finding “Instructor Issues,” the faculty team 
produced a series of recommendations to address 
instructor attitudes, instructor practices and course 
inconsistencies, quality of instructors, and spoken 
language. To address “instructor attitudes,” the faculty 
team recommended that: (a) deans and chairs follow up 
with the student complaint process, (b) complaint offices 
be located on all campuses, (c) faculty have a presence in 
the grievance office, (d) cultural sensitivity be addressed 
in professional development, and (e) faculty focus 
groups be conducted to clarify HERI Faculty Survey 
results. To address “instructor practices and course 
inconsistencies,” the faculty team recommended that 


instructors: (a) provide e-mail solutions, (b) provide 
handouts on how to use campus voicemail system, 

(c) post and emphasize faculty contact information, 

(d) ensure that syllabi indicate when assignments can 
be expected to be returned to students, and (e) provide 
various opportunities for feedback on assignments. 

To address “quality of instructors,” the faculty team 
recommended that: (a) student evaluation questions 
be incorporated in the quality evaluation rubric for the 
Faculty Search Committee Protocol, (b) instructors be 
made aware of expectations for quality, (c) instructors 
encourage questions in the classroom, and (d) instructors 
demonstrate and stimulate interest in the subject matter. 
Finally, to address “spoken language and accents,” the 
faculty team recommended that instructors: (a) make 
lessons understandable and accessible to all students, 

(b) summarize the main points of all lessons, and 

(c) provide visuals. 

Beyond offering recommendations to address “instructor 
issues,” the faculty team designed and implemented 
a follow-up study focusing on the “instructor issues” 
barrier. The purpose of the study was to: (a) re-examine 
the student-faculty interaction variables found in the 2007 
focus group study; (b) discover quantitative values of the 
five disaggregated categories ranking highest within the 
“instructor issues” barrier, (c) use a survey approach to 
design item groupings to analyze mean scores at the item, 
scale, and dimension levels, (d) use ordinal-level data to 
run potential tests for statistical difference and correlation 
relationships among student characteristics, (e) confirm 
or reject the strength of 2007 barriers study findings, 
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and (f) establish the current state of student-faculty 
interactions with regard to “instructor issues,” specifically 
the five disaggregated categories. The instrument used in 
the follow-up study is found in Appendix 1. 

Conclusion 

National philanthropic organizations, such as Lumina 
Foundation for Education and the Bill & Melinda Gates 
Foundation, and President Obama have set bold goals 
for doubling the numbers of young Americans who attain 
a postsecondary degree. Foundations and the Obama 
Administration see community colleges as key to meeting 
this challenge. The goal stated in the introduction of 
increasing community college graduation rates by five 
percent would be a step in the right direction. In fact, we 
need more than that if we are to meet the demands of a 
knowledge-based economy and offer more individuals and 
families an opportunity to achieve the American Dream. 


We have learned a great deal in the first five years of the 
Achieving the Dream initiative. Most importantly, we 
learned that a 5 percent increase is not only possible, 
but also probable if colleges can use data in meaningful 
and productive ways. More than that, we have witnessed 
promising, data-driven practices that are on track to 
increasing graduation rates by much more than 5 percent. 
This guide is offered as one tool, among many, to help 
colleges collect, analyze, and use data in ways that can 
lead to increases in student success. 


Kenneth P. Gonzalez is University Professor of Education 
at the University of San Diego and a data facilitator for 
the Achieving the Dream initiative. 
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APPENDIX 1 Survey Instrument for follow-up study on “instructor issues” 


Barriers Revisited Study 

Your opinion is important to us. 

Your Campus: What grade do you expect in this course: 

How many credit hours have you completed: Are you an online only student: Yes No (Circle) 

Are you taking Developmental Courses: Yes No (Circle) 

In this section, please rate the following statements: 




Strongly 

Somewhat 

Somewhat 

Strongly 

1. 

IPCI Grouping 

agree 

agree 

disagree 

disagree 

1. 

Instructors provide timely feedback 

□ 

0 

B 

□ 

2. 

Instructors are available outside of class 

a 

□ 

□ 

□ 

3. 

Instructors respond when I try to contact them 

□ 

□ 

□ 

□ 

4. 

Instructors explain grading 

□ 

B 

B 

□ 

5. 

Instructor's assignments are related to the course 

□ 

B 

B 

□ 


Comments for Questions 1 -5: 


II. 

Ql Grouping 

Strongly 

agree 

Somewhat 

agree 

Somewhat 

disagree 

Strongly 

disagree 

6. 

Instructors present materials in an interesting way 
for learning 

□ 

B 

B 

□ 

7. 

Instructors encourage students to participate in class 

□ 

B 

B 

□ 

8. 

Instructors are knowledgeable in their subjects 

□ 

B 

B 

□ 

9. 

Instructors have skills necessary to teach courses in major 

□ 

B 

B 

□ 

10. 

Instructors answer students’ questions well 

□ 

B 

B 

□ 


Comments for Questions 6-10: 
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III. ISLA Grouping 

Strongly 

agree 

Somewhat 

agree 

Somewhat 

disagree 

Strongly 

disagree 

1 1 . Instructors make time to summarize main points of lesson 

□ 

□ 

□ 

□ 

1 2. Instructors use words that are clear where I can 
understand material 

□ 

E3 

□ 

□ 

13. Instructors do not have language barriers making it 
difficult to learn 

□ 

□ 

□ 

□ 

14. No communication problems exist 

El 

□ 

□ 

□ 

1 5. Instructors address questions from students clearly 

□ 

□ 

□ 

□ 

Comments for Questions 11-15: 





IV. IE Grouping 

Strongly 

agree 

Somewhat 

agree 

Somewhat 

disagree 

Strongly 

disagree 

1 6. Instructors expectations are not fair 

□ 

□ 

□ 

□ 

1 7. Instructors do not provide enough time in class to teach 
material 

□ 

□ 

□ 

□ 

1 8. The tests do not cover information that I was asked 
to study 

a 

□ 

□ 

□ 

19. Instructors teach at too high of a level 

a 

0 

B 

□ 

20. Instructors expect us to learn too much of the material 
on our own 

□ 

0 

B 

□ 


Comments for Questions 1 6-20: 
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V. IA Grouping 

21 . Instructors are interested in academic problems 
of students 

22. Instructors treat students fairly with respect 

23. Sometimes instructors are rude and embarrass you 

24. Instructors are willing to help 

25. Instructors provide support you need to succeed 
in college 


Strongly 

Somewhat 

Somewhat 

Strongly 

agree 

agree 

disagree 

disagree 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

El 

□ 

□ 

□ 

□ 

B 

□ 

□ 

□ 

B 

□ 

□ 


Comments for Questions 21-25: 


Exit Comment #1 

In your experiences at STC, do instructors contribute to retention (keeping students enrolled)? Yes No (Circle) 
How? 


Exit Comment #2 

In your experiences at STC, do instructors contribute to student withdrawal (barriers that influence students dropping)? 

Yes No (Circle) 

How? 
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